KW&SACHUSETTS INSTITUTE Cf TECHNOLOGY 



ARTIFICIAL INTELLIGENCE LABORATORY 



Artilicial intelligence May 1975 

Memo No. 333 



ON VISUAL DETECTION OF LIQrfT SOURCES 



Shimon Ulrnan 



AJvH^i 



The paper addresses (he following problem; Given an array ol l<ght intensities obtjurted from 
soma scene-, find (he. light sagrc-tt in tfc* original scene. The following <aclD-s are discussed 
from the po n! oF view d1 Iheir relevance 10 lighl sources de'ediOn: The highest inlensity irt 
this siene, absdule mlensily <j*\m, local ?nd g.obal contrast, comparison with (he »ver,a£e 
intensity, and lightness computation. They are shown to be insufficient for explaining humans" 
.ihility to identily light sOurrs^ n their visual firld. Finally, a method I Or iKxOT.plisliing. the 
source detorliyn task in the mOndriain wflrlct is ptesentBd, 
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©n V^ual Detection of Light Sources. 

1 ■ The Problem. 

U It is sometimes illustrative to chinjc of some of [he research done in Lower Le*ek Vision 

as aimed as, answering- the following question; Suppose you are given the diagram 
illustrated below, winch is an array of light intensities taken from lome picture. The 
question LS; what Le there^ 

4? 74 60 I2L 106 71 23 70 5* 17 Ii2 3fl 7D 123 100 44 103 65 42 40 104 
^ 35 20 12L llfi 13"! ST 50 16 L25 37 32 54 64 25 40 105 73 67 77 105 3 
47 22 70 114 hi H 30 16 55 43 5? ft ]1& &4 32 +3 43 57 13 65 71 14 I? 
S3 LI3 127 [7 75 33 42 32 76 76 J30 56 15 23 22 56 72 ]L4 24 lib 70 131 
9 12 ft LI 23 &3 47 73 IEI 6L 4? 65 |3J 74 57 26 130 122 41 125 122 127 
121 34 63 M2 66 22 20 El [23 40 52 30 L4 27 36 64 22 57 13 115 35 74 
115 123 122 55 72 LL4 24 115 70 121 12 115 35 64 25 20 55 12 73 13 23 fi5 
47 73 111 61 42 55 I3I 74 57 126 130 LEE HI 125 122 127 123 4L 15 102 33 
47 77 7.5 113 102 42 105 32 30 1C0 103 jJ4 50 121 36 J2L 37 20 106 111 115 
23- 45 42 74 60 55 77 71 23 76 64 47 112 30 70 123 JOG 44 103 66 42 36 
45 35 20 121 L16 m 37 30 45 125 37 32 54 64 25 40 105 72 67 77 &3 9L 
12 73 123 123 63 47 73 ILL 61 42 65 I3L 74 57 2E |30 122 14] 125 122 ffl 

The correctness of theanS-Wertc the question "What Ls thej* r is (o be determined by 
its consistency with people's response 10 the same question, when presented with the 
corresponding pLttuie, The main point :s toemphasiie that the only data available to the 
vlsuat system J* an array of light intensities, or perhaps a few or them for the different 
kinds of reKprors fch che- eye. 

In the pai titular problem addressed herp. \W general problem "What is there?" is 
replaced by "where are the light sources in the picture" As the answer we are after is to be 
consistent with human Competency Lt is natural to as* whether people are indeed capable of 



detecting light sources in 'heir visual f ields. The answer Is that in many cases they 
undoubtedly are. For example, in mast cases it ii easy to tell whether a light bulb is turned 
on or off. A simpfe demonstration of this ability ii The use of brakes-light in automobiles as 
a Waning system. This is not to 43^ that huminstah always detect light sources correctly/. 
On the contrary; we cannot detect a weak light source if the background is loo bright, e.g 
5t*r J in daytime., while on the other hand, areas which are not actual light sources are 
sometimes, perceived as such, as in the case of the retro-reflecting shining signs on the 
highway. However, the interesting point, as we shiU see, is not that wt occasionally make 
erroneous judgments, bur, that in many situations We make the right ones. 

The problem of deLecLlng light sources has been confined here to "Achromatic 
Mondrians". By an Achromatic Mondnan we mean an array of rectangular shapes., of 
different. si;es, and different levels of hUr-k, e;rey and white, ai in picture £. The term 
"rVtondrlam" for such arrays was used by E. Land and j, McC-ann, [Land 19TJ] due to their 
resemblance to the painting* by Piet Mondrian. These Mondrians serve to simplify the 
environment, especially by excluding colors, (Evans, in [Evans JtfH] has seme work on the 
contribution of colors to the perception of fluorescence.) and by discarding both shadings 
and the type of "f uziy' light source, shown in picture I. 

The Mondilan is composed of pieces of paper, glued- together onto a while, 
transparent, background sheet. It is then placed on a thJck piece of cardboard. By making 
a hole in the cardboard, and placing it above a fluorescent lamp, (be area above the hole 
becomes, a uniform light source. 

A subject is then presented with the MondrLan. and asked whether he detects any 



light source m it. and where The question; we raise are. under what conditions hill the 
subject be able to detect light sources in ihe Mondhan.and by whai possible methods can 
juch a. task be accomplished. 
& Six possibilities, and a discussion af their insufficiency. 

We shall proceed by examining the following six factors from the point of View of 
their relevance to light-sources detection. 

1. The highest intensity Ln the v jj.ua I field. 

2. H]£h absolute intensity value. 

3. Local contrast. 
1. Global contrast. 

5. Intensity compared with the average intensity in the scene. 
G. Lightness computation. 

We shall Conclude that even in the Simple case qf the Mcmdrian; these factors are not 
sufficient to account for the ability of human iubjecu tn detect light sources, in their visual 
field. The following notations are used in the sequel: The light falling on the Mondrian Is 
called the WurtmuiBTi, and js denoted by T, The radiant area in the Mondnan is denoted 
by "A*; the light transmit red. through it is calted the MuTce-intenslty. and is denoted by "L". 
2.1 The highest intensity in the scene, 

A perceived light source is not necessarily associated with the highest Ugh* intensity 
in the field. Suppose, for example, that we create an illumination gradient, and at the place 
where the intensity Ls low, we place a wea*. light source. The intensities can be set in such a 
way, that the light source will be perceivable, while the intensity graph will look tike that in 



figure I; 

The intensity at B is, higher 
than the intensity at A. It is, in 
fact, the highest intensity in the 
scene. Skill, only A is perceived 
as a light source. 

FtgUri L 

Thus, having the highest intensjry in the scene is neither a necessary, nor u it, 
obviously a, sufficient condition, for being perceived as a light soUroe. 
2.2 High absolute intensity value. 

A light source is not necessarily associated with an area of high absoluie intensity. 
Against a diik tnouijh background a firefly Ls perceived as a light source, although the 
absolute light intensity coming From it is very low. We shall resume this subject of absolute 
intensities after describing the rtSUl&JOf two Simple experiment which will liplp to elucidate 
our point ai well a serve as an introduction to the nest section. 

Experiment L [f, at a given illumination I. to* begin with a very low soutxe-iriTen sity and 
gradually increase it. the following happens. At first, the source is nor detectable. When it 
grows stronger, :her'p is a srange of uncertainty, and then, finally, it becomes prominent. Th* 
range of uncertainty is bounded by two ''thresholds*; the lower Threshold is the source 
intensity for which the subject suspects for the first time there might be a light source there; 
the upper threshold IS the source intensity at which he becomes sure. This distinction is 
important when carrying out eitperlrMntS with a "jJUSt noticeable* light source, as it specifies 



the experimental condition more precisely 

Experiment 2: [f the source ini-erk&Ltjr IS SST ju-5t above «he upper threshold, and we 

gradually Jncreate the illumination, the reverse process tates plate, via the Upper threshold, 

Che range of uncertainty, and the tower threshold, to a situation where the source Is no 

longer detects ble- 

In this last experiment where L was constant and I increased, the total intensity 

coming from are* A increased, while at the am? time it ceased to appear as a light source. 

Tht conclusion Is that It was not the high intensity value it A that made it appear radiant. 

Rjather, it seems that the major role was played by the ratio between the intensity at A and 

that of the surrounding's- We shall turn therefore to consider this ratio. 

2-3 Contrail: 

A variable that seems suitable to explain the above results, Jl the contrast. The 
contrait between intensity l> and intensity it can be defined as (Ii - 1*) { (Ii + U). As- this 
contrast is a monotonic function of li/li. (assuming li/li it positive) high contrast simply 
corresponds to 3 i bemg "many times" greater than It. 

We can now explain che experiments described above in terms of contrast- Suppose 
we have an illumination I - lCOuniu on some arbitrary scale. L ^ 100 units; and instead of 
a Mondrian we have a uniform iurface with reflectance - D.5 (that is. it reflects 50 of the 
the incident light.) The light intensity coming; from A will be 100 (source intensify) ■ 100 * 
0.5 < reflection), that is* 150 units. From the surroundings, we gee only the 50 units of 
reflected light, so that the ratio is 3;l> or the contrast is 0.5 if che illumination is raised to 
1000 unit*, che new mteniitiei ratio between A and the surroundings will he only 6;5V 



Although the light intensity at A is higher this lime, the eonrrast li much lower, ofily l/IL 

A plausible conclusion from the dLscusswn so far is that whenever the conn rait 
between a stimulus A and its surroundings exceeds a Certain value h the former is perceived 
as a light source. There is a kind of "rationale" !0 SU<h a conclusion- Surfaces, in nature do 
not usually have reflectivity values approaching the extreme* or LOO or D . A very high 
contrasr value is thus not likely, to be achieved by reflectance change* alone; rather* it might 
indicate the presence of a light source. 

A question: of interest at this point is "just how high K high". Namely, how high are 
the contrasts found in natural scenes without light scuicfs, and what contrast is needed for a 
light source: to be just noticeable (that is. just above ihe lower threshold). These values were 
measured in a scene containing pieces of a very white paper, a very dark paper, a strip of 
shadow, and a light source at the tower threshold Intensity. The measurement? were 
performed using the A.I. Vidiccm r ffOrtt which the real intent uy values, are difficult to 
obtain. The following results should serve, Therefore, more as indications of the relative 
magnitudes than as accurate absolure values. 

The following ratios of intensity values were computed: 
White to black ratio: G-S Whll€ (0 shadow ratio: 13-ft 

Source to black ratio: 11-15 Source to shadow ratio: 19-23 

In contrast with these high values, a light source was also perceivable in a picture 
where the ratio of intensities was in no place higher than 15. Thus, the perception of a. 
light MUree cannot be identified with that af a hegh enough contrast, While a high enough 
contrast seems, as far as I have been able to determine, sufficient to induce the perception Of 



a light source, it is not a necessary condition. ]n other words, the presence of a light source 
can be "'deduced' From factors) Other than contrast. The rest of the paper is an attempt to 
find such a factor. What makes the question intriguing is ihe Tact that (here docs not se*m 
to be an Obvious candidate for Che tail. 

We proceed by discussing some of the more immediate candidates for a solution, and 
show their inadequacy. They are; global as opposed to local contrast, average illumination, 
and lightness computation. 
2-4 Global versus local contrast: 

Consider the situation in' picture % The source intensity was set at the lower 
threshold. Picture 3 is of the same S-etting. only this time With a dark lltl surrounding the 
light source, The contrast hetiveen the source and its immediate surroundings is thus 
multiplied by 6-9 This change In contrast did not, however, make the light SOUnce more 
noticeable. The following claim can be rjiwd here, the change Ln contrast between pictures 
2 and 3 h only in the immediate contrast, that is, between the light source and its immediate 
surroundings. The global contrast, namely between the light source and the darkest area in 
the field, had however not been effected. I therefore repeated this experiment,, this, time 
changing the global surroundings as well, and with similar results. Using the above 
distinction it can noiv be stated, that light sources are sometimes detectable when both the 
g loba] and the local comrasts were low, ihcrefoie nehhe? is a necessary condition for the 
source detection. 
2,5 Average illumination: 

Another factor that had been considered is the influence of changes in rhe average 



Illumination. WJH the Threshold* of detection become lower, if vt list, for example, a darker 
Monti nan (while maintaining both global and immediate contrasts fixed)? No such 
influence has been detected. 
*-6 Lightness computation: 

Finally, let me turn, to a brief analysis, of the above results in terms of hft-hrnp» To see 
whether this approach provides us with the key to the direction of lawcontrait sources. 
The computation of lightness from intensities (SeeiUnd 197]]. [Horn I9M]) involves the 
separation of sharp intensity change* from gradual ones.. Thus, an intensny graph like the 
one shown m figure £A, will he decomposed into a sharp-changes component {figure £B) 
and a gradual-change component (figure 2C). 





Figure 2A Figure 2S Figure 2C 

One might try to use thli deccrtttposition to explain the t9.it that A is perceived as a 
light source, although the intensity at & l& higher if the lightness contrast IS computed 
instead of the intensity contrast, then area A obviously gets the highest value. Examination 
Of the experimental data reveals, however, that the computation: of consrasr vJa lightness, 
d«i not help solving the basic problem contrast it still not a necessary condition. The 



visual svsiem somehow distinguishes between possible interpretation* of the scene: a contrast 

created by light-and-dark surfaces on the one hand. and a contrast created by the presence 

of a light source on the other. To main* the problem of 'mere thin one possible 

interpret ion" clearer, consider the following intensity array: 

44 43 43 42 42 4MI 40 40 310 LOT L03 107 LOS lOS 10* 109 102 101 100 SO 49 AS 4? 46 
44 43 4S 42 42 41 4L 40 40 110 LOT 106 107 LOT 105 104 103 102 101 IQO 50 19 46 47 46 
44 43 43 42 42 4L 4L 40 40 ILO LOT LOG 107 LOT 105 I04 LOG L02 10L 100 &0 49 46 47 46 
44 43 43 42 42 4HI 40 4Q I ID L09 JOE 1Q7 LOB 1C5 304 LOT LD2 10L 100 50 49 4& 4"? 46 

The corresponding lightness mains discards slow changes and therefore will look like thLs; 

SO 50 30 SO 30 30 SO SO 30 10O 100 LOT LOT LOT lOT LOT 100 100 100 100 50 50 W 50 &Q 
30 30 30 30 30 30 50 30 30 LOT LOO LOO 100 LOO LOT lOT 100 100 100 100 50 50 50 50 50 
30 30 30 30 30 30 30 30 30 LOT LOT LOO LOT LOT 100 100 300 100 100 100 50 50 50 SO 50 
SO 30 30 30 SO SO SO SO SO LOT LOT LOO LOO 100 100 100 10O LOT IOC LOO 50 50 W 50 50. 

Ln both cases, there is, more than a single possible Interpretation. One Interpretation 
id that the area* simply have different ref lectivlttes. Thus, the central area can be 
normalized to 1,0, and the left and right areas assigned the values of 0.3 and 4.5 respectively, 
A different interpretation is that the right area has a reflectivity of |.0, the leFt - 0.&, and the 
cen:er is a li^ht source. 

To be sure, if we 'zftivt the real reflectance o; one area,, we could have determined (at 
least in the case of Mondrlans) all the reflectance value* m the picture, and then we could 
have also assigned the Label "light source" to areas of reflectance greater then one The only 
trouble with this method ii that there 1 does not seem to exist a way of determining real 
reflectance values. Still, in many cases the visual system is abk to maVe (he right 
interpretation of the scene, In a picture rather similar to the array presented above, but 
with even less contrast, a light source had been perceived. That is P the visual system 



managed to somehow pick (correct^} the light-source interpretation. How ran this be done? 
The following seetLeri proposes a method, 

5+ The proposed method. 

$.1 General description of the method. 

Roughly speaking, the proposed method is the following; 
Given two adjacent areas, compute both their intensity-ratio and their gradient-ratio, and 
compare the two. If the ratio* are not equal, on* of The areas is a light source. 

It Is based on the following fwo observations: 
I. All that is needed tor the detection of a light source, is thecoma values of the reflectance 
ratios in the scene. 

Z In many cases this real refleoa nee ratio can be computed^ even in the presence of a light 
source, by comparing intensity gradients. 

Let me discuss each in turn. 

I, Using (he value of (rjnfc 

The preceding section seemed to imply that a correct normalisation of the 
reflectances is needed in order to detect lighT sources, b^ identifying them with areas, of 
reflectance greater than I. It turns out, howe-ver, that we don't need that r->uch. 

Consider two adjacent areas: area no and area. no. Land suppose that wc somehow 
know the reflectance raciq rs/n. Assuming the illumination Li a conlinuoui function of rlie 
position, we can then determine whether one of the above areas is a light source- The 
continuity of I implies :hat the illumination on the- two sides of the borderline between the 



areas is approximately equal, IT i Hi e mpajurerwnts are taken dose Enough to the borderline. 
From this equality it follows that if neither of ihfr areas is a light JOUr«, the intensity ratio 
should equal the ref Iterance, ratio. Hence. iP the ratio values do not agree,, we tan deducs 
the presence of a light source and, at it turns out, we can also compute the actual source 

intensity. 

2, Computing (r^i) from gradients: 

As had been mentioned above, the only values permitted in the computation arc the 
intensity values at different points in the visual field. The way used boLh by Land and 
Horn to determine reflectance ratios from these intensities is basically the following: Find 
th<p peine* of sharp changes in the intensity distribution, like point A In figure % 
Then conclude that k tmtmtty 

the- rate :><\ is- 
equal to the tatio eufei 
when *i ii measured to 
the left of th,E "jump" A r 

Figure $ 
and ei Is measured to the right of it. This computation, however, presupposes the existence 
of no light sources in the picture. For Otherwise Jr. might oe the case that, Tor example, r* - 
n r but There is a light source at area which creates the intensity jump at A. That is CO *ay f 
in order to compute the reflectances ratio we need first to discover the light sources, and in 
order to discover the light sources ^e need the reflectance fltios! Here 15 ii way nut of this 
circle. Suppose, (and this is usually the case), that the illumination is not absolutely uniform. 




but has a gradient which aver seme i^e* is more or less, linear, say a gradient of ICO 
intensity units to an inch, When such an ilium ination n reflected from a surface of 
refkuancc ft. not wily is the overall intensity reduced by a factor of K; the gradient l will 
be ft times the original one as welL 

In figure 4, the reflectances are taken to be ]/2 In Area and 1/4 in area I. 
Note that not only js lifh - 2, intensity 

but S*/$» also ii equal »2; 
where S* is trie slope of the 
intensity graph row wired. 
from A to the left, and 
Sb (s the slope measured from B to 
the right. 

Figure 4 

If we now add! a uniform Itght source to area 0, the ratio UIH will change, but the 
ratio SVSt will remain the same, and Still equal to rtfri. S* and Sn car), of course, be 
computed from the Intensity values alone. 

Thus, we Can Use the gradient ratio 10 compute the reflectance ratio, and then use 
thii ratio as shown in the preceding section to test for the pretence of a light source. Mexi, 
we shall describe how the compilation it actually implemented. 
3-2 The "SoUTCe Operator" and Lts implementation. 

Fe* notations: as before, 1 denote* the illumination, L - the source intensity (the 
source is, assumed ro be in area 0). S is the gradient, r denote* the reflectance and e standi 




for the inrermTy of the light reaching the eye. Noie that e ii the only parameter we ire 
alfowed to measure. The di&cuiiion so Tar shows chat from the obvious relation: 
H} es - (I* *n)tL 

we can deduce that upcler "imOGCn" illumination; 
(2) L»e»'iH(S(/S]) 

if the points and I, where the rneasuremen ts. are taken, are dose enough to each other, 
However", "real life" measurements fanfl" to look more like figure 5. 
In this case we can iAtgnmy ^ 

no longer suppose 
that I* Is, equal to It, 
as they are too far 
apart Rather, it is: 
h-U*Kd 

Figure 5 
Where K is the LntentLty gradient in intensity uniR per unit distance, and d is the distance 

of B fram A. Sd: :nig with thfl underlying equation (i), L - e* - rv <* la, we now get: 
L * fig ■* ri * (Ik - Kd) - et - dm - r^sKd 
eu ■ ei n (r#/ri) + S«d (Since rsK -Si) 
This test equation* wj^h the slopes replacing the reflectances 
(S) L - v: -'.■: > (S^Sl) - Sad 
will be referred to as the "S-Operator" 




It is a rather Jtraigh (forward task 10 implement [he S-Gperator and run it on actual 
intensity array*, Several detail* hai/e to be added, mainly to cape with the problems of noise 
and insufficient resolution of the Vidicnn. For example,, the right-slope at r>o I ii C i cannot be 
computed sjmply by, jay, Km) - I(i): rather Some averaging is needed. In the program thii 
average is taken over a distance which varies with the situation Another eSAmple is. the 
treatment of irregular pcnnts t whose intensities are exceptionally higher or lower than their 
surrounding. It jo happens that in most cases, they are lower, due perhaps CO dust on the 
lenses. 

Figure 6 at the end of this section shows the intensity distribution oh rained from a 
Kene which contained a light source tn the right of point A. Figure 1 shows the 
corresponding Output of the 5-Operator. As negative S-values are meaningless tn the 
particular algorithm that had been used, ihe OUtpmt C*n be taken IS the rnaKimum of and 
the computed S-values, as in fifllt* A. The "Source value' is the highest at the real source 
borderline. Its relative- magnitude IS, enough to allow the setting of some threshold, above 
which an area can be labeled as a light source. In the general case, this threshold has to 
depend on the overall intensity. An &-va1ue of 100 units, which is not significant when the 
intensity is around 5000 uniis and wish a nolie level of ~CC, is significant indeed when the 
intensity is aiOUtid 100 and the nmse Lev el around 10 units. This fat* contributes to the 
phenomenon that a light source, whkh ii detectable at low illuminations, beeemei 
undeiwtabk at higher ones. 

A final remark The above method fSl-Ses the possibility Of detecting a "dark light 
source". An area with an average intensity: of, say, 0.S that of its surround trigs,, might be 



"declared" as a lighr source if the light |jadieni [here is only, say. O.t [he jr radient of the 
surroundings, Js it possible to COm[njtL a situation In which an area with hghl intensities 
[Oiver than the surroundings will be perceived as a light source? Althgugh some experiments 
have been carried, Cot, I don't yet know the answer to this interesting quernon 
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